Measuring Large Overlay Networks - The Overnet Example

Peer-to-peer overlay networks have grown significantly in size and sophistication over the last years. Meanwhile, distributed hash tables (DHT) provide efficient means to create global scale overlay networks on top of which various applications can be built. Although filesharing still is the most prominent example, other applications are well conceivable. In order to rationally design such applications, it is important to know (and understand) the properties of the overlay networks as seen from the respective application. This paper reports the results from a two week measurement of the entire Overnet network, the currently most widely deployed DHT-based overlay. We describe both, the design choices that made that measurement feasible and the results from the measurement itself. Besides the basic determination of network size, node availability and node distribution, we found unexpected results for the overlay latency distribution. 1 Description of Overnet and Kademlia Overnet is a popular peer-to-peer filesharing network, currently used by more than 10 unique servants (i.e. server-clients). It is based on Kademlia [8], a general peer-to-peer routing algorithm that can be used to implement distributed hash tables (DHT). Its service is similar to that of other popular DHT routing algorithms like Chord [15], CAN [11], or Pastry [14]. In all these DHTs, participating nodes are assigned a unique identifier, usually a 128 or 160 bit long unsigned integer. In Overnet, the ID is 128 bits long, generated randomly (uniformly) during first node startup, and saved permantently for consecutive starts. These identifiers can be viewed as addresses in an overlay network that extends the functionality of the underlying network infrastructure [3]. Unlike classical networks, these overlays are organized in a way that every address maps to a node participating in the overlay network. Thus, these addresses can be used for hashing arbitrary keys to the network’s nodes. Hence, the term distributed hash table. With Kademlia, this address-to-node mapping assigns each address the closest node where closeness is defined by the XOR-metric, i. e. d(a, b) = a⊕ b. This metric has the benefit of symmetry, i. e. when node B is close to node A, then also node A is close to node B. In order to provide routing in this address space, Kademlia nodes have to maintain overlay links to other Kademlia nodes. To this end, all nodes have socalled buckets each holding k different (Kademlia-address, IP-transport-address) pairs representing these overlay links. The assignment of pairs to the buckets is based on Kademlia-address prefixes: For each (shortest) prefix that is not a prefix of the respective node’s address, there is one bucket. As an example, a node with ID 110100 would maintain a bucket for nodes whose addresses are starting with 0, 10, 111, 1100, 11011 and 110101. (Clearly, some of these buckets will be empty, since typically for large prefixes no appropriate overlay links are available.) For routing, a message then simply is forwarded to one of the peers from the longest prefix bucket. Overnet employs this to create a filesharing network. We do not address any of the aspects associated with that particular use, but rather study the properties of the Overnet network resulting from this general overlay mechanism. Our results can therefore be expected to reflect fundamental aspects of such large overlay networks. This paper is structured as follows: After discussing related work in section 2, we describe our measurement setup in section 3. We especially focus on general issues that are likely to occur with future peer-to-peer protocols, too. Section 4 summarizes our measurement results. Besides the determination of the fundamental parameters of the Overnet network, we report some astonishing findings which we cannot fully explain at the moment. In section 5, we conclude with an outlook to future work.